1. Field of the Invention
The present invention relates to improvements in neural networks, hardware for carrying out the functions of a neural network, neural network processors, and neural network pattern recognition apparatuses.
2. Description of the Related Art
A neural network, namely, a system of recognizing predetermined input information and providing the results of the recognition which is, based on a conception entirely differing from those on which conventional methods are based, has been developed and applied to various fields. The neural network is a model of a human brain, which can be realized in various ways.
A neural network has been proposed in a mathematical algorithm having a complex structure. Accordingly, conventional neural networks have been realized by computer simulation. Computer simulation, however, operates at a comparatively low processing speed, which is a problem in some practical applications. Recently, a comprehensive study of the neural network has been made and pieces of hardware for realizing neural networks have been proposed. However, the proposed hardware deals with neural networks having only one or two layers.
A Neocognitron is one model of a neural network. Only a few studies have been made on the development of hardware for realizing a Neocognitron, since a Neocognitron is a neural network having a complex structure. Hardware for realizing a Neocognitron has been reported in a paper published by MIT. This paper was published in the poster session the NIPS (Neural Information Processing & Systems) '90 conference. The hardware is simple in structure comprising, in combination, 143 CCD arrays and seven MDACs (multiplier DA converters). Most circuits employed in the hardware are digital circuits. Basically, both input data and coefficient data are stored in the digital circuits, and the semianalog MDACs carry out multiplication. The method of fabricating this system, however, has not been able to fabricate division circuits satisfactorily to date only first layer has been realized. This hardware has a small degree namely, integration of seven multipliers in 29 MM.sup.2.
The realization of such a neural network in hardware has encountered many difficulties and hence methods have been studied for the high-speed simulation of a neural network having three or more layers. One of the methods simulates a neural network using a program executed by parallel processing computers. However, if this method is employed, it often occurs that the computational topology of the neural network does not coincide with the architecture of each computer which reduces the efficiency of data transmission between the processing elements. Even if parallel computers having many processing elements are employed for high-speed simulation, it is difficult to improve the cost performance.